<!doctype html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><meta name=robots content="noodp"><meta http-equiv=x-ua-compatible content="IE=edge, chrome=1"><title>ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded - Yulin Lewis' Blog</title><meta name=keywords content="雨临Lewis,Java,hugo,hexo,博客"><meta name=Description content="不想当写手的码农不是好咸鱼_(xз」∠)_"><meta property="og:title" content="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded"><meta property="og:description" content="问题与分析
在本地测试无误后将ELK部署到了测试环境，结果第二天发现Logstash挂掉了，一开始以为是自动部署之类导致的问题。之后几天时间里Logstash总是会down掉，查看了下日志文件，发现报错如下：


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


[2019-06-28T07:56:13,148][FATAL][logstash.runner          ] An unexpected error occurred!
{
	: error=>#<Errno: : EPIPE: Brokenpipe-<STDOUT>>,
	: backtrace=>[&#34;org/jruby/RubyIO.java:1457:in `write'&#34;,
	&#34;org/jruby/RubyIO.java:1428:in `write'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-stdout-3.1.4/lib/logstash/outputs/stdout.rb:43:in `block in multi_receive_encoded'&#34;,
	&#34;org/jruby/RubyArray.java:1734:in `each'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-stdout-3.1.4/lib/logstash/outputs/stdout.rb:42:in `multi_receive_encoded'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/outputs/base.rb:87:in `multi_receive'&#34;,
	&#34;org/logstash/config/ir/compiler/OutputStrategyExt.java:114:in `multi_receive'&#34;,
	&#34;org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:97:in `multi_receive'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:390:in `block in output_batch'&#34;,
	&#34;org/jruby/RubyHash.java:1343:in `each'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:389:in `output_batch'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:341:in `worker_loop'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:304:in `block in start_workers'&#34;]
}"><meta property="og:type" content="article"><meta property="og:url" content="https://lewky.cn/posts/4d3f2358.html/"><meta property="og:image" content="https://lewky.cn/logo.png"><meta property="article:published_time" content="2019-08-08T20:11:45+08:00"><meta property="article:modified_time" content="2019-08-08T20:11:45+08:00"><meta name=twitter:card content="summary_large_image"><meta name=twitter:image content="https://lewky.cn/logo.png"><meta name=twitter:title content="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded"><meta name=twitter:description content="问题与分析
在本地测试无误后将ELK部署到了测试环境，结果第二天发现Logstash挂掉了，一开始以为是自动部署之类导致的问题。之后几天时间里Logstash总是会down掉，查看了下日志文件，发现报错如下：


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


[2019-06-28T07:56:13,148][FATAL][logstash.runner          ] An unexpected error occurred!
{
	: error=>#<Errno: : EPIPE: Brokenpipe-<STDOUT>>,
	: backtrace=>[&#34;org/jruby/RubyIO.java:1457:in `write'&#34;,
	&#34;org/jruby/RubyIO.java:1428:in `write'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-stdout-3.1.4/lib/logstash/outputs/stdout.rb:43:in `block in multi_receive_encoded'&#34;,
	&#34;org/jruby/RubyArray.java:1734:in `each'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-stdout-3.1.4/lib/logstash/outputs/stdout.rb:42:in `multi_receive_encoded'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/outputs/base.rb:87:in `multi_receive'&#34;,
	&#34;org/logstash/config/ir/compiler/OutputStrategyExt.java:114:in `multi_receive'&#34;,
	&#34;org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:97:in `multi_receive'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:390:in `block in output_batch'&#34;,
	&#34;org/jruby/RubyHash.java:1343:in `each'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:389:in `output_batch'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:341:in `worker_loop'&#34;,
	&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:304:in `block in start_workers'&#34;]
}"><meta name=application-name content="雨临Lewis的博客"><meta name=apple-mobile-web-app-title content="雨临Lewis的博客"><meta name=theme-color content="#ffffff"><meta name=msapplication-TileColor content="#da532c"><link rel="shortcut icon" type=image/x-icon href=/favicon.ico><link rel=icon type=image/png sizes=32x32 href=/favicon-32x32.png><link rel=icon type=image/png sizes=16x16 href=/favicon-16x16.png><link rel=apple-touch-icon sizes=180x180 href=/apple-touch-icon.png><link rel=mask-icon href=/safari-pinned-tab.svg color=#5bbad5><link rel=manifest href=/site.webmanifest><link rel=canonical href=https://lewky.cn/posts/4d3f2358.html/><link rel=prev href=https://lewky.cn/posts/b37842a8.html/><link rel=next href=https://lewky.cn/posts/d5221ccd.html/><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/normalize.css@8.0.1/normalize.min.css><link rel=stylesheet href=/css/style.min.css><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free@5.13.0/css/all.min.css><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/animate.css@3.7.2/animate.min.css><script type=application/ld+json>{"@context":"http://schema.org","@type":"BlogPosting","headline":"ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded","inLanguage":"","mainEntityOfPage":{"@type":"WebPage","@id":"https:\/\/lewky.cn\/posts\/4d3f2358.html\/"},"genre":"posts","keywords":"ELK系列, Logstash","wordcount":811,"url":"https:\/\/lewky.cn\/posts\/4d3f2358.html\/","datePublished":"2019-08-08T20:11:45+08:00","dateModified":"2019-08-08T20:11:45+08:00","license":"This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.","publisher":{"@type":"Organization","name":"雨临Lewis","logo":"https:\/\/lewky.cn\/images\/avatar.jpg"},"author":{"@type":"Person","name":"雨临Lewis"},"description":""}</script></head><body header-desktop=fixed header-mobile=auto><script type=text/javascript>(window.localStorage&&localStorage.getItem('theme')?localStorage.getItem('theme')==='dark':('auto'==='auto'?window.matchMedia('(prefers-color-scheme: dark)').matches:'auto'==='dark'))&&document.body.setAttribute('theme','dark');</script><div id=mask></div><div class=wrapper><header class=desktop id=header-desktop><div class=header-wrapper><a href=https://github.com/lewky class=github-corner target=_blank title="Follow me on GitHub" aria-label="Follow me on GitHub"><svg width="3.5rem" height="3.5rem" viewBox="0 0 250 250" style="fill:#70b7fd;color:#fff;position:absolute;top:0;border:0;left:0;transform:scale(-1,1)" aria-hidden="true"><path d="M0 0 115 115h15l12 27L250 250V0z"/><path d="M128.3 109C113.8 99.7 119 89.6 119 89.6 122 82.7 120.5 78.6 120.5 78.6 119.2 72 123.4 76.3 123.4 76.3 127.3 80.9 125.5 87.3 125.5 87.3 122.9 97.6 130.6 101.9 134.4 103.2" fill="currentcolor" style="transform-origin:130px 106px" class="octo-arm"/><path d="M115 115C114.9 115.1 118.7 116.5 119.8 115.4L133.7 101.6C136.9 99.2 139.9 98.4 142.2 98.6 133.8 88 127.5 74.4 143.8 58 148.5 53.4 154 51.2 159.7 51 160.3 49.4 163.2 43.6 171.4 40.1 171.4 40.1 176.1 42.5 178.8 56.2 183.1 58.6 187.2 61.8 190.9 65.4 194.5 69 197.7 73.2 200.1 77.6 213.8 80.2 216.3 84.9 216.3 84.9 212.7 93.1 206.9 96 205.4 96.6 205.1 102.4 203 107.8 198.3 112.5 181.9 128.9 168.3 122.5 157.7 114.1 157.9 116.9 156.7 120.9 152.7 124.9L141 136.5C139.8 137.7 141.6 141.9 141.8 141.8z" fill="currentcolor" class="octo-body"/></svg></a><div class=header-title><a href=/ title="Yulin Lewis' Blog"><span class=header-title-pre><i class="fas fa-fw fa-atom"></i></span>雨临Lewis</a></div><div class=menu><div class=menu-inner><div class=dropdown><a href=/posts/ class="menu-item menu-more dropbtn" title=点击查看所有文章><i class="fas fa-fw fa-archive"></i>归档</a><div class="menu-more-content dropdown-content"><a href=/categories/><i class="fas fa-fw fa-th"></i>分类 </a><a href=/tags/><i class="fas fa-fw fa-tag"></i>标签 </a><a href=/hot/ title=文章热度Top15><i class="fas fa-fw fa-fire"></i>热度 </a><a href=/donation/ title=感谢打赏，老板大气~><i class="fas fa-fw fa-donate"></i>打赏 </a><a href=/about/><i class="fas fa-fw fa-at"></i>关于</a></div></div><a class=menu-item href=/friends/ title=欢迎申请友链><i class="fas fa-fw fa-link"></i>友链</a><div class=dropdown><a href=javascript:void(0); class="menu-item menu-more dropbtn" title=一些有意思的东东~><i class="fas fa-fw fa-fan fa-spin"></i>趣味</a><div class="menu-more-content dropdown-content"><a href=/funny/high/ title="前方高能♂ 小心外放！" target=_blank rel=noopener><i class="fas fa-fw fa-dizzy"></i>燥起来！ </a><a href=/funny/mikutap/ title=初音未来音乐游戏 target=_blank rel=noopener><i class="fas fa-fw fa-music"></i>在线打碟 </a><a href=/funny/catch-the-cat/ title=逮住那只猫!><i class="fas fa-fw fa-cat"></i>抓住猫咪</a></div></div><div class=dropdown><a href=/ class="menu-item menu-more dropbtn" title=回到首页><i class="fas fa-fw fa-atom"></i>站点</a><div class="menu-more-content dropdown-content"><a href=/bbs/ title=来留言吧~><i class="fas fa-fw fa-comment"></i>公告留言 </a><a href=https://seo.chinaz.com target=_blank rel=noopener><i class="fas fa-fw fa-chart-line"></i>SEO查询 </a><a href=https://www.ping.cn/http/lewky.cn target=_blank rel=noopener><i class="fas fa-fw fa-bezier-curve"></i>网站测速 </a><a href=/posts/e62c38c4.html/><i class="fas fa-fw fa-cog fa-spin"></i>建站日志</a></div></div><div class=dropdown><a href=javascript:void(0); class="menu-item menu-more dropbtn"><i class="fas fa-fw fa-book"></i>文档</a><div class="menu-more-content dropdown-content"><a href=https://gohugo.io/documentation/ target=_blank rel=noopener><i class="fas fa-fw fa-star"></i>Hugo文档 </a><a href=https://javanote.doc.lewky.cn/ title=尚在完善中~ target=_blank rel=noopener><i class="fab fa-fw fa-java"></i>Java 笔记</a></div></div><a class=menu-item href=/search/ title=渲染搜索结果需要等待几秒钟~><i class="fas fa-fw fa-search"></i>搜索
</a><span class="menu-item delimiter"></span><a href=https://travellings.link target=_blank class=menu-item rel=noopener title=开往-友链接力><i class="fas fa-fw fa-subway"></i></a><a href=https://rssblog.vercel.app/ class=menu-item target=_blank rel=noopener title=RSSBlog><i class="fas fa-fw fa-inbox"></i></a><a href=https://github.com/lewky class=menu-item target=_blank rel=noopener title=GitHub><i class="fab fa-fw fa-github"></i></a></div></div></div></header><header class=mobile id=header-mobile><div class=header-container><div class=header-wrapper><div class=header-title><a href=/ title="Yulin Lewis' Blog"><span class=header-title-pre><i class="fas fa-fw fa-atom"></i></span>雨临Lewis</a></div><div class=menu-toggle id=menu-toggle-mobile><span></span><span></span><span></span></div></div><div class=menu id=menu-mobile><div class=dropdown><a href=/posts/ class="menu-item menu-more dropbtn" title=点击查看所有文章><i class="fas fa-fw fa-archive"></i>归档</a><div class="menu-more-content dropdown-content"><a href=/categories/><i class="fas fa-fw fa-th"></i>分类 </a><a href=/tags/><i class="fas fa-fw fa-tag"></i>标签 </a><a href=/hot/ title=文章热度Top15><i class="fas fa-fw fa-fire"></i>热度 </a><a href=/donation/ title=感谢打赏，老板大气~><i class="fas fa-fw fa-donate"></i>打赏 </a><a href=/about/><i class="fas fa-fw fa-at"></i>关于</a></div></div><a class=menu-item href=/friends/ title=欢迎申请友链><i class="fas fa-fw fa-link"></i>友链</a><div class=dropdown><a href=javascript:void(0); class="menu-item menu-more dropbtn" title=一些有意思的东东~><i class="fas fa-fw fa-fan fa-spin"></i>趣味</a><div class="menu-more-content dropdown-content"><a href=/funny/high/ title="前方高能♂ 小心外放！" target=_blank rel=noopener><i class="fas fa-fw fa-dizzy"></i>燥起来！ </a><a href=/funny/mikutap/ title=初音未来音乐游戏 target=_blank rel=noopener><i class="fas fa-fw fa-music"></i>在线打碟 </a><a href=/funny/catch-the-cat/ title=逮住那只猫!><i class="fas fa-fw fa-cat"></i>抓住猫咪</a></div></div><div class=dropdown><a href=/ class="menu-item menu-more dropbtn" title=回到首页><i class="fas fa-fw fa-atom"></i>站点</a><div class="menu-more-content dropdown-content"><a href=/bbs/ title=来留言吧~><i class="fas fa-fw fa-comment"></i>公告留言 </a><a href=https://seo.chinaz.com target=_blank rel=noopener><i class="fas fa-fw fa-chart-line"></i>SEO查询 </a><a href=https://www.ping.cn/http/lewky.cn target=_blank rel=noopener><i class="fas fa-fw fa-bezier-curve"></i>网站测速 </a><a href=/posts/e62c38c4.html/><i class="fas fa-fw fa-cog fa-spin"></i>建站日志</a></div></div><div class=dropdown><a href=javascript:void(0); class="menu-item menu-more dropbtn"><i class="fas fa-fw fa-book"></i>文档</a><div class="menu-more-content dropdown-content"><a href=https://gohugo.io/documentation/ target=_blank rel=noopener><i class="fas fa-fw fa-star"></i>Hugo文档 </a><a href=https://javanote.doc.lewky.cn/ title=尚在完善中~ target=_blank rel=noopener><i class="fab fa-fw fa-java"></i>Java 笔记</a></div></div><a class=menu-item href=/search/ title=渲染搜索结果需要等待几秒钟~><i class="fas fa-fw fa-search"></i>搜索</a>
<a href=https://travellings.link target=_blank class=menu-item rel=noopener title=开往-友链接力><i class="fas fa-fw fa-subway"></i></a><a href=https://rssblog.vercel.app/ class=menu-item target=_blank rel=noopener title=RSSBlog><i class="fas fa-fw fa-inbox"></i></a><a href=https://github.com/lewky class=menu-item target=_blank rel=noopener title=GitHub><i class="fab fa-fw fa-github"></i></a></div></div></header><div class="search-dropdown desktop"><div id=search-dropdown-desktop></div></div><div class="search-dropdown mobile"><div id=search-dropdown-mobile></div></div><main class=main><div class=container><div class=toc id=toc-auto><h2 class=toc-title>目录</h2><div class=toc-content id=toc-content-auto></div></div><article class="page single"><h1 class="single-title animated flipInX">ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded</h1><div class=post-meta><div class=post-meta-line><span class=post-author><a href=/ title=Author rel=author class=author><i class="fas fa-user-circle fa-fw"></i>雨临Lewis</a></span>&nbsp;<span class=post-category>收录于 <a href=/categories/elk/><i class="far fa-folder fa-fw"></i>ELK</a></span></div><div class=post-meta-line><i class="far fa-calendar fa-fw"></i>&nbsp;<time datetime=2019-08-08>2019-08-08</time>&nbsp;<i class="far fa-calendar-plus fa-fw"></i>&nbsp;<time datetime=2019-08-08>2019-08-08</time>&nbsp;<i class="fas fa-pencil-alt fa-fw"></i>&nbsp;约 811 字
<i class="far fa-clock fa-fw"></i>&nbsp;预计阅读 2 分钟&nbsp;<span id=/posts/4d3f2358.html/ class=leancloud_visitors data-flag-title="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded">
<i class="far fa-eye fa-fw"></i>&nbsp;<span class=leancloud-visitors-count></span>&nbsp;次阅读
</span>&nbsp;
<a href=#comments id=post-meta-vcount title=查看评论><i class="fas fa-comment fa-fw"></i>&nbsp;<span id=/posts/4d3f2358.html/ class=waline-comment-count></span>&nbsp;条评论</a></div></div><div class="details toc" id=toc-static kept><div class="details-summary toc-title"><span>目录</span>
<span><i class="details-icon fas fa-angle-right"></i></span></div><div class="details-content toc-content" id=toc-content-static><nav id=TableOfContents><ol><li><a href=#问题与分析>问题与分析</a></li><li><a href=#解决方案>解决方案</a></li></ol></nav></div></div><div class=content id=content><h2 id=问题与分析>问题与分析</h2><p>在本地测试无误后将ELK部署到了测试环境，结果第二天发现Logstash挂掉了，一开始以为是自动部署之类导致的问题。之后几天时间里Logstash总是会down掉，查看了下日志文件，发现报错如下：</p><div class=highlight><div class=chroma><table class=lntable><tr><td class=lntd><pre class=chroma><code><span class=lnt> 1
</span><span class=lnt> 2
</span><span class=lnt> 3
</span><span class=lnt> 4
</span><span class=lnt> 5
</span><span class=lnt> 6
</span><span class=lnt> 7
</span><span class=lnt> 8
</span><span class=lnt> 9
</span><span class=lnt>10
</span><span class=lnt>11
</span><span class=lnt>12
</span><span class=lnt>13
</span><span class=lnt>14
</span><span class=lnt>15
</span><span class=lnt>16
</span><span class=lnt>17
</span></code></pre></td><td class=lntd><pre class=chroma><code class=language-json data-lang=json><span class=p>[</span><span class=mi>2019-06-28</span><span class=err>T</span><span class=mi>07</span><span class=err>:</span><span class=mi>56</span><span class=err>:</span><span class=mi>13</span><span class=p>,</span><span class=mi>148</span><span class=p>][</span><span class=err>FATAL</span><span class=p>][</span><span class=err>logstash.runner</span>          <span class=p>]</span> <span class=err>An</span> <span class=err>unexpected</span> <span class=err>error</span> <span class=err>occurred!</span>
<span class=p>{</span>
	<span class=err>:</span> <span class=err>error=&gt;#&lt;Errno:</span> <span class=err>:</span> <span class=err>EPIPE:</span> <span class=err>Brokenpipe-&lt;STDOUT&gt;&gt;,</span>
	<span class=err>:</span> <span class=err>backtrace=&gt;[</span><span class=nt>&#34;org/jruby/RubyIO.java:1457:in `write&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;org/jruby/RubyIO.java:1428:in `write&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-stdout-3.1.4/lib/logstash/outputs/stdout.rb:43:in `block in multi_receive_encoded&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;org/jruby/RubyArray.java:1734:in `each&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/vendor/bundle/jruby/2.3.0/gems/logstash-output-stdout-3.1.4/lib/logstash/outputs/stdout.rb:42:in `multi_receive_encoded&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/outputs/base.rb:87:in `multi_receive&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;org/logstash/config/ir/compiler/OutputStrategyExt.java:114:in `multi_receive&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:97:in `multi_receive&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:390:in `block in output_batch&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;org/jruby/RubyHash.java:1343:in `each&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:389:in `output_batch&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:341:in `worker_loop&#39;&#34;</span><span class=p>,</span>
	<span class=nt>&#34;/home/cbx6/software/logstash-6.6.1/logstash-core/lib/logstash/pipeline.rb:304:in `block in start_workers&#39;&#34;</span><span class=err>]</span>
<span class=p>}</span>
</code></pre></td></tr></table></div></div><p>从堆栈信息里可以看到关键字眼：<code>block in multi_receive_encoded</code>，<code>block in output_batch</code>；另外，还可以发现这些错误信息都是由<code>logstash-output-stdout-3.1.4</code>这个插件引发的。</p><p>简单分析来看，应该是由于测试环境下同一时间内太多message要经由<code>logstash-output-stdout</code>输出到控制台造成的某种未知的并发问题。下面是对应的Logstash的output的配置：</p><div class=highlight><div class=chroma><table class=lntable><tr><td class=lntd><pre class=chroma><code><span class=lnt> 1
</span><span class=lnt> 2
</span><span class=lnt> 3
</span><span class=lnt> 4
</span><span class=lnt> 5
</span><span class=lnt> 6
</span><span class=lnt> 7
</span><span class=lnt> 8
</span><span class=lnt> 9
</span><span class=lnt>10
</span></code></pre></td><td class=lntd><pre class=chroma><code class=language-xml data-lang=xml>output {
    stdout {
        codec =&gt; rubydebug
    }
    elasticsearch {
        hosts =&gt; [&#34;{{ cbx_logstash_es_server }}&#34;]
        index =&gt; &#34;%{indexName}&#34;
        action =&gt; &#34;index&#34;
    }
}
</code></pre></td></tr></table></div></div><p>根据配置，并结合堆栈信息来分析，可以认为是Logstash的stdout插件在高并发状态下使用<code>rubydebug</code>进行编解码时抛出了异常。</p><p>其实这里的stdout插件是不必要的，之前只是在本地测试使用到的。而在测试环境下，并发量远非本地测试能比，此外将大量的message输出到console上也会对性能产生影响。可以说，这种配置等同于在Java代码中频繁使用<code>System.out.print()</code>语句来输出信息，并不推荐这种做法。</p><h2 id=解决方案>解决方案</h2><p>将配置文件里的stdout插件去掉，最终output的配置如下：</p><div class=highlight><div class=chroma><table class=lntable><tr><td class=lntd><pre class=chroma><code><span class=lnt>1
</span><span class=lnt>2
</span><span class=lnt>3
</span><span class=lnt>4
</span><span class=lnt>5
</span><span class=lnt>6
</span><span class=lnt>7
</span></code></pre></td><td class=lntd><pre class=chroma><code class=language-xml data-lang=xml>output {
    elasticsearch {
        hosts =&gt; [&#34;{{ cbx_logstash_es_server }}&#34;]
        index =&gt; &#34;%{indexName}&#34;
        action =&gt; &#34;index&#34;
    }
}
</code></pre></td></tr></table></div></div><p>将stdout插件的配置去除后，在之后的一段时间里，测试环境的Logstash不再发生异常退出，证实该issue确实是由<code>stdout</code>的<code>codec</code>所引发的。注意，不要在正式环境使用该插件来输出信息到控制台，有可能会引发类似的并发异常问题或者性能问题。</p><div class="details admonition warning open"><div class="details-summary admonition-title"><i class="icon fas fa-exclamation-triangle fa-fwwarning"></i>警告<i class="details-icon fas fa-angle-right fa-fw"></i></div><div class=details-content><div class=admonition-content>本文最后更新于 <span class=timeago datetime=2019-08-08T20:11:45 title="August 8, 2019">August 8, 2019</span>，文中内容可能已过时，请谨慎使用。</div></div></div><div class=post-reward><input type=checkbox name=reward id=reward hidden>
<label class=reward-button for=reward>赞赏支持</label><div class=qr-code><label class=qr-code-image for=reward><img class=image src=https://cdn.jsdelivr.net/gh/lewky/lewky.github.io@master/images/common/wechat.png>
<span>微信打赏</span></label>
<label class=qr-code-image for=reward><img class=image src=https://cdn.jsdelivr.net/gh/lewky/lewky.github.io@master/images/common/alipay.png>
<span>支付宝打赏</span></label></div></div></div><div class=post-footer id=post-footer><div class=post-info><div class=post-info-line><div class=post-info-mod><span>更新于 2019-08-08</span></div><div class=post-info-license><span><a rel="license external nofollow noopener noreffer" href=https://creativecommons.org/licenses/by-nc/4.0/ target=_blank>CC BY-NC 4.0</a></span></div></div><div class=post-info-line><div class=post-info-md><span><a class=link-to-markdown href=/posts/4d3f2358.html/index.md target=_blank>阅读原始文档</a></span></div><div class=post-info-share><span><a href=javascript:void(0); title="分享到 Twitter" data-sharer=twitter data-url=https://lewky.cn/posts/4d3f2358.html/ data-title="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded" data-hashtags=ELK系列,Logstash><i class="fab fa-twitter fa-fw"></i></a><a href=javascript:void(0); title="分享到 Facebook" data-sharer=facebook data-url=https://lewky.cn/posts/4d3f2358.html/ data-hashtag=ELK系列><i class="fab fa-facebook-square fa-fw"></i></a><a href=javascript:void(0); title="分享到 Hacker News" data-sharer=hackernews data-url=https://lewky.cn/posts/4d3f2358.html/ data-title="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded"><i class="fab fa-hacker-news fa-fw"></i></a><a href=javascript:void(0); title="分享到 Line" data-sharer=line data-url=https://lewky.cn/posts/4d3f2358.html/ data-title="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded"><i data-svg-src=https://cdn.jsdelivr.net/npm/simple-icons@2.14.0/icons/line.svg></i></a><a href=javascript:void(0); title="分享到 微博" data-sharer=weibo data-url=https://lewky.cn/posts/4d3f2358.html/ data-title="ELK系列(7) - 测试环境下Logstash异常退出：block in multi_receive_encoded"><i class="fab fa-weibo fa-fw"></i></a></span></div></div></div><div class=post-info-more><section class=post-tags><i class="fas fa-tags fa-fw"></i>&nbsp;<a href=/tags/elk%E7%B3%BB%E5%88%97/>ELK系列</a>,&nbsp;<a href=/tags/logstash/>Logstash</a></section><section><span><a href=javascript:void(0); onclick=window.history.back();>返回</a></span>&nbsp;|&nbsp;<span><a href=/>主页</a></span></section></div><div class=post-nav><a href=/posts/b37842a8.html/ class=prev rel=prev title="ELK系列(5) - Logstash怎么分割字符串并添加新的字段到Elasticsearch"><i class="fas fa-angle-left fa-fw"></i>ELK系列(5) - Logstash怎么分割字符串并添加新的字段到Elasticsearch</a>
<a href=/posts/d5221ccd.html/ class=next rel=next title="Linux - /bin/sh^M: bad interpreter: No such file or directory">Linux - /bin/sh^M: bad interpreter: No such file or directory<i class="fas fa-angle-right fa-fw"></i></a></div></div><div id=comments><div id=waline></div><script src=https://cdn.jsdelivr.net/npm/@waline/client/dist/Waline.min.js></script><script>new Waline({el:'#waline',meta:["nick","mail","link"],requiredMeta:["nick","mail"],login:"force",placeholder:"为防恶意灌水攻击，评论前需注册并登录，望见谅~",serverURL:"https://comment.lewky.cn/",avatarCDN:"https://sdn.geekzu.org/avatar/",pageSize:20,avatar:"retro",lang:"zh-CN",visitor:true,highlight:true,uploadImage:false,emoji:['https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/嘉然今天吃什么','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/大航海嘉然','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/向晚大魔王','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/贝拉kira','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/珈乐Carol','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/乃琳Queen','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/EveOneCat','https://cdn.jsdelivr.net/gh/walinejs/emojis@1.0.0/weibo','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/滑稽','https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/emoji/default']});</script></div></article></div></main><footer class=footer><div class=footer-container><div class=footer-line><span id=run-time></span>|
<a href=http://rssblog.vercel.app/ target=_blank rel=noopener title=RSSBlog><i class="fas fa-fw fa-inbox"></i>&nbsp;RSSBlog</a></div><div class=footer-line><i class="far fa-copyright fa-fw"></i><span itemprop=copyrightYear>2018 - 2022</span><span class=author itemprop=copyrightHolder>&nbsp;<a href=/ target=_blank>雨临Lewis</a></span>&nbsp;|&nbsp;<span class=license><a rel="license external nofollow noopener noreffer" href=https://creativecommons.org/licenses/by-nc/4.0/ target=_blank>CC BY-NC 4.0</a></span><span class=icp-splitter>&nbsp;|&nbsp;</span><br class=icp-br><span class=icp><a target=_blank href=http://www.beian.miit.gov.cn/ style=font-weight:700>粤ICP备19103822</a></span></div></div></footer></div><div id=fixed-buttons><a href=# id=back-to-top class=fixed-button title=回到顶部><i class="fas fa-arrow-up fa-fw"></i></a><a href=# id=view-comments class=fixed-button title=查看评论><i class="fas fa-comment fa-fw"></i></a></div><div class=sidebar_wo><div id=leimu><img src=https://cdn.jsdelivr.net/gh/lewky/lewky.github.io@master/images/b2t/leimuA.png alt=雷姆 onmouseover="this.src='https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/b2t/leimuB.png'" onmouseout="this.src='https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/b2t/leimuA.png'" title=回到顶部></div><div class=sidebar_wo id=lamu><img src=https://cdn.jsdelivr.net/gh/lewky/lewky.github.io@master/images/b2t/lamuA.png alt=雷姆 onmouseover="this.src='https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/b2t/lamuB.png'" onmouseout="this.src='https:\/\/cdn.jsdelivr.net\/gh\/lewky\/lewky.github.io@master/images/b2t/lamuA.png'" title=回到底部></div></div><link rel=stylesheet href=https://cdn.jsdelivr.net/npm/lightgallery.js@1.2.0/dist/css/lightgallery.min.css><script type=text/javascript src=https://cdn.jsdelivr.net/npm/smooth-scroll@16.1.3/dist/smooth-scroll.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/lazysizes@5.2.2/lazysizes.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/lightgallery.js@1.2.0/dist/js/lightgallery.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/lg-thumbnail.js@1.2.0/dist/lg-thumbnail.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/lg-zoom.js@1.2.0/dist/lg-zoom.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/clipboard@2.0.6/dist/clipboard.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/sharer.js@0.4.0/sharer.min.js></script><script type=text/javascript>window.config={"code":{"copyTitle":"复制到剪贴板","maxShownLines":30},"comment":{},"lightGallery":{"actualSize":false,"exThumbImage":"data-thumbnail","hideBarsDelay":2000,"selector":".lightgallery","speed":400,"thumbContHeight":80,"thumbWidth":80,"thumbnail":true}};</script><script type=text/javascript src=/js/theme.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/jquery@2.1.3/dist/jquery.min.js></script><script type=text/javascript src=https://cdn.jsdelivr.net/npm/jquery-backstretch@2.1.18/jquery.backstretch.min.js></script><script>var $cdnPrefix="https://cdn.jsdelivr.net/gh/lewky/lewky.github.io@master";</script><script type=text/javascript src=/js/custom.js></script><script>var _hmt=_hmt||[];(function(){var hm=document.createElement("script");hm.src="https://hm.baidu.com/hm.js?6f278ed0fdb01edb3b1e7398379e5432";var s=document.getElementsByTagName("script")[0];s.parentNode.insertBefore(hm,s);})();</script></body></html>